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DESCRIPTION 

Color Balance Adjustment Referencing Specific Color Reproduction 

TECHNICAL FIELD 
5 This invention relates to technology for adjusting color balance in 

images. 

BACKGROUND ART 
Processing known as color balance adjustment can be performed on 
the entirety of an image taken by an image generation device such as a digital 
10 still camera or a video camera (for example, see JP-2001-320727-A). 

Color balance adjustment allows for a reduction of color cast in the 
image, so that photographic subjects that were originally white are correctly 
reproduced as white. 

However, color balance adjustment sometimes results in undesired 
15 changes to the colors of specific photographic objects. For example, if near- 
white pixels have become bluish, color balance adjustment decreases the blue 
component in the overall image, so that the red component is accentuated. 
This presents a problem in that, if skin color in the image is at the desired 
color before color balance adjustment, the skin color will become more red, 
20 which is to say that the color will be less desirable. This problem of changing a 
desired color to a less desirable color as a result of color balance adjustment is 
not limited to skin color, but may also occur for other chromatic colors such as 
green and blue, and is, in general, a common problem for specific chromatic 
colors. 

25 

DISCLOSURE OF THE INVENTION 
An object of the present invention is to provide technology capable of 
mitigating the phenomenon wherein color balance adjustment has the opposite 
effect of shifting a color close to a specific chromatic color in an image away 
30 firom the desired color. 

An image processing device according to an aspect of the present 
invention is an image processing device for adjusting color balance in an image 
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to be processed, comprising^ a specific color pixel ratio calculation module 
configured to calculate a ratio of specific color pixels having a color within a 
first predetermined range in the vicinity of a specific chromatic color, fi-om 
among all of the pixels present in an image to be processed; a specific color 
5 pixel average calculation module configured to calculate an average value of 
pixel values for the specific color pixels; a color balance correction value 
calculation module having a first computation mode for calculating a color 
balance correction value based on the specific color pixel ratio and the average 
pixel value for the specific color pixels; and a color balance adjustment 

10 execution module configured to execute the color balance adjustment for the 
entirety of the image to be processed, using the color balance correction value. 

By virtue of this image processing device, the color balance correction 
value is calculated based on the specific color pixel ratio and the average pixel 
value of the specific color pixels, whereby it is possible to mitigate the 

15 phenomenon wherein color balance adjustment has the opposite effect of 

shifting a color close to a specific chromatic color in an image away from the 
desired color as a result. 

It is preferable that the color balance correction value calculated in 
the first computation mode includes a component that increases with increases 

20 in a difference between a predetermined specific color target pixel value for the 
specific chromatic color and the average pixel value for the specific color pixels, 
and that increases with increases in the specific color pixel ratio. 

By virtue of this constitution, it is possible to keep the colors of pixels 
having a color close to the specific chromatic color at suitable colors. 

25 The image processing device may further comprise- a high luminance 

pixel average pixel value calculation module configured to calculate an average 
value of pixel values for a plurality of high luminance pixels which have colors 
in a predetermined second color range having low-color-saturation and high- 
luminance, among all of the pixels present in the image to be processed, 

30 wherein the color balance correction value calculation module may"(i) execute 
computations in the first computation mode if the average value of hue values 
of the plurality of high luminance pixels is not within a predetermined range 
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in the vicinity of the specific chromatic color, and (ii) execute computations in a 
second computation mode for calculating the color balance correction value 
using the average pixel value of the plurality of high luminance pixels, without 
using the specific color pixel ratio and the average pixel value of the specific 
5 color pixels, if the average value of the hue values of the plurality of high 
luminance pixels is within the predetermined range in the vicinity of the 
specific chromatic color. 

By virtue of this constitution, the color balance correction value can 
be set to a suitable value, according to whether or not the average hue value of 

10 the high luminance pixels is within the predetermined range in the vicinity of 
the specified chromatic color. Specifically, in the first computation mode, pixels 
having colors in the vicinity of the specific chromatic color can be maintained 
at suitable colors; and in the second mode, a desirable color balance can be 
achieved for the entire image to be processed. 

15 Alternatively, in the image processing device, the specific chromatic 

color may be skin color, and an image data file representing the image to be 
processed includes photographic scene type information representing a type of 
photographic scene in the image to be processed, wherein the color balance 
adjustment execution module may- (i) execute computations in the first 

20 computation mode if the photographic scene type information indicates 
portrait; and (ii) execute computations in a second computation mode for 
calculating the color balance correction value using the average pixel value of 
the plurality of high luminance pixels, without using the specific color pixel 
ratio and the average pixel value of the specific color pixels, if the photographic 

25 scene type information indicates a scene other than portrait. 

By virtue of this constitution, if the photographic scene type indicates 
portrait, pixels having colors in the vicinity of skin color can be maintained at 
suitable colors. Meanwhile, if the photographic scene type indicates a scene 
other than portrait, a desirable color balance can be achieved for the entire 

30 image to be processed. 

An image processing device according to another aspect of the present 
invention is an image processing device for adjusting color balance in an image 
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to be processed, comprising- a specific color pixel average calculation module 
for calculating an average value for pixel values of specific color pixels having 
a color within a first predetermined range in the vicinity of a specific chromatic 
color, from among all of the pixels present in the image to be processed; a color 
5 balance correction value calculation module having a first computation mode 
for calculating a color value correction value based on the average pixel value 
of the specific color pixels; and a color balance adjustment execution module 
configured to execute the color balance adjustment for the entirety of the 
image to be processed, using the color balance correction value. 

10 By virtue of this image processing device, the color balance correction 

value is calculated based on the average pixel value of the specific color pixels, 
whereby it is possible to mitigate the phenomenon wherein color balance 
adjustment has the opposite effect of shifting a color close to a specific 
chromatic color in an image away from the desired color. 

15 It is preferable that the color balance correction value calculated in 

the first computation mode includes a component that increases with increases 
in a difference between a predetermined specific color target pixel value for the 
specific chromatic color and the average pixel value of the specific color pixels. 

By virtue of this constitution, it is possible to keep the colors of pixels 

20 having a color close to the specific chromatic color at suitable colors. 

The image processing device may further comprise a high luminance 
pixel average pixel value calculation module configured to calculate an average 
value of pixel values for a plurality of high luminance pixels which have colors 
in a predetermined second color range having low-color-saturation and high- 

25 luminance, among all of the pixels present in the image to be processed, 

wherein the color balance correction value calculation module may- (i) execute 
computations in the first computation mode if the average value of hue values 
of the plurality of high luminance pixels is not within a predetermined range 
in the vicinity of the specific chromatic color; and (iO execute computations in a 

30 second computation mode for calculating the color balance correction value 

using the average pixel value of the plurality of high luminance pixels, without 
using the average pixel value of the specific color pixels, if the average value of 
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the hue values of the plurality of high luminance pixels is within the 
predetermined range in the vicinity of the specific chromatic color. 

By virtue of this constitution, the color balance correction value can 
be set to a suitable value, according to whether or not the average value of the 
5 hues of the high luminance pixels is within the predetermined range in the 
vicinity of the specific chromatic color. Specifically, in the first computation 
mode, pixels having colors in the vicinity of the specific chromatic color can be 
maintained at suitable colors! and in the second mode, a desirable color 
balance can be achieved for the entire image to be processed. 

10 Alternatively, in the image processing device, the specific chromatic 

color may be skin color; and an image data file representing the image to be 
processed may include photographic scene type information representing a 
type of photographic scene in the image to be processed; wherein the color 
balance adjustment execution module may" (i) execute computations in the 

15 first computation mode if the photographic scene type information indicates 
portrait; and (ii) execute computations in a second computation mode for 
calculating the color balance correction value using the average pixel value of 
the plurality of high luminance pixels, without using the average pixel value of 
the specific color pixels, if the photographic scene t3rpe information indicates a 

20 scene other than portrait. 

By virtue of this constitution, if the photographic scene type indicates 
portrait, pixels having colors in the vicinity of skin color can be maintained at 
suitable colors. Meanwhile, if the photographic scene type indicates a scene 
other than portrait, a desirable color balance can be achieved for the entire 

25 image to be processed. 

Note that the present invention can be carried out in various modes; 
for example, it may be carried out in such modes as an image processing 
method and an image processing device, a computer program for implementing 
the functions of this method or this device, and a recording medium on which 

30 this computer program is recorded. 
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FIG. 1 is a schematic view illustrating the structure of an image 
processing system according to one embodiment of the present invention. 

FIG. 2 is a schematic view illustrating the structure of an image data 

file MF. 

5 FIG. 3 is a block diagram illustrating the structure of a color balance 

adjustment module 212. 

FIG. 4 is a schematic view illustrating an image to be processed TMG 
and various color balance adjustment parameters therefor. 

FIG. 5 is a schematic view illustrating one example of a high 
10 luminance color range HLR. 

FIG. 6 is a flow chart illustrating the processing procedure for color 
balance adjustment in a first embodiment. 

FIG. 7 is the graph illustrating color balance adjustments performed 
using color balance correction values AC(R), AC(G) and AC(B). 
15 FIG. 8 is a flow chart illustrating the processing procedure for color 

balance adjustment in a second embodiment. 



BEST MODE FOR CARRYING OUT THE INVENTION 
Preferred embodiments of the present invention are described in the 
20 . following order- 

A. First Embodiment 

B. Second Embodiment 

C. Variants 



25 A. First Embodiment 

FIG. 1 is a schematic view illustrating the structure of an image 
processing system according to one embodiment of the present invention. The 
system comprises a digital camera 100 as an image generation device, a 
computer 200 as an image processing device and a printer 300 as an image 

30 output device. An image data file MF generated by the digital camera 100 is 
sent to the computer 200 or to the printer 300 for output. Note that, in the 
present specification the term "output" encompasses display on a monitor and 
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printing. 

FIG. 2 is a schematic view illustrating the structure of the image data 
file MF. This image data file MF comprises photography information, main 
image data MM and thumbnail image data TM. The main image data MM and 
5 the thumbnail image data TM are usually in a compressed data format (for 
example JPEG data compression). Note that it is acceptable for the thumbnail 
image data TM not to be included in the image data file MF. 

The photography information includes the various conditions and 
settings for the time at which the image data file MF was generated by the 

10 digital camera 100. In FIG. 2, "photographic scene type" is shown as an 

example of photography information. The photographic scene tj^e can be set to 
"portrait," "landscape," "night scene" or the like. 

A printer driver 210, which serves to generate printing data based on 
the image data file MF, is installed on the computer 200 (FIG. 1). 

15 The printer driver 210 comprises a color balance adjustment module 

212, a color conversion module 214, a halftone processing module 216, and an 
output processing module 218. The printing data PD generated by the printer 
driver 210 is supplied to the printer 300 fi-om the output processing module 
218. The printing data PD comprises dot data that indicates the ink dot 

20 recording conditions for each pixel in the main scanning line having a printing 
resolution, and sub-scan feed data that specifies the sub-scan feed amount. 
Note that the printer driver 210 corresponds to a computer program for 
realizing the functions that generate dot data for printing. Furthermore, the 
color balance adjustment module 212 corresponds to a computer program for 

25 realizing the functions that perform color balance adjustment. 

The program for realizing the functions of the printer driver 210 can 
be supplied in the form of a program recorded on a computer readable 
recording medium. Various computer readable media can be used as this 
recording medium, such as flexible disks and CD-ROMs, magneto optical disks, 

30 IC cards, ROM cartridges, punch cards, and printed material on which 
symbols such as bar codes are printed. 

FIG. 3 is a block diagram illustrating the structure of a color balance 
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adjustment module 212. FIG. 4 is a schematic view illustrating an example of 
an image to be processed TMG and various color balance adjustment 
parameters. The color balance adjustment module 212 comprises the following 
modules. 

5 (l) High luminance pixel average pixel value calculation module 220: 

This module 220 calculates the following average pixel values for high 
luminance pixels based on the data for the image to be processed Dorg, which 
corresponds to main image data MM in FIG. 2. 

a. Average color component values: CB_ave(R), CB_ave(R), CB_ave(B); 
10 b. Average luminance value: CB_ave(Y); and 

c. Average hue value: CB_ave(H) 

The term "high luminance pixels" refers to pixels having a color 
within a color range, or a high luminance color range, which has low color 
15 saturation values and high luminance values. FIG. 5 shows one example of a 
high luminance color range HLR. Note that various formulas can be used to 
calculate the luminance Y and the color saturation S of each pixel; for example, 
the following formulas can be used. 

Y=0.587xG+0. 1 14XB+0.299XR . . . (l) 

20 S=V(a*2+b*2) ...(2) 

In the formula, a* and b* are coordinates in the La*b* color system. 
The lightness value L, from the La*b* color system, as well as the brightness 
values or lightness values from other color spaces, can be used as the 
25 luminance value Y. Likewise, color saturation values from other color spaces 
such as the HSL (Hue/Saturation/Luminance) color space, or the HSB 
(Hue/Saturation/Brightness) color space, can be used for the color saturation 
value S. 

The average color component values CB_ave(R), CB_ave (G) and 
30 CB_ave (B) are average values for each of the RGB color components. 

Furthermore, the average luminance value CB__ave(Y) is the average value for 
the luminance Y, and to the average hue value CB_ave(H) is the average value 
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for the hue H. The angle of the a* axis in the La*b* color system, or the hue 
values H from the HSL color space or the HSB color space may, for example, 
be used for the hue value H. 



5 (2) Skin color pixel ratio calculation module 222: 

This module 222 calculates the ratio of skin color pixels SPratio, 
based on the data for the image to be processed Dorg. The ratio of skin color 
pixels SPratio is given by the following formula (3). 

SPratio = [skin color pixel count]/[total pixel count] ...(3) 

10 

Whether or not a given pixel is a skin color pixel is determined 
according to whether or not the color of that pixel is within a predetermined 
skin color range. The skin color range can, for example, be determined based 
on a hue H range and a lightness L range. 

15 

(3) Skin color pixel average pixel value calculation module 224^ 

This module 224 calculates the average pixel values SC_ave(R), 
SC_ave(G) and SC_ave(B) for a plurality of skin color pixels, in terms of each 
of the RGB color components, based on the data for the image to be processed 
20 Dorg. 

(4) Color balance correction value calculation module 226^ 

This module 226 calculates the color balance correction values AC(R), 
AC(G) and AC(B) in terms of each of the RGB color components, based on the 
25 various values described above. In the present embodiment, the color balance 
correction values AC(R), AC(G) and AC(B) are calculated according to the 
following formulas (4a) to (4c). 

AC(R)=klx{CB^ave(Y) - CB,ave(R)} 

+k2x{SC_target(R) ■ SC_ave(R)}xSPratio ...(4a) 
30 AC(G)=klx{CB_ave(Y) • CB_ave(G)} 

+k2x{SC_target(G) - SC_ave(G)} xSPratio ...(4b) 
AC(B)=klx{CB_ave(Y) • CB_ave(B)} 

-9- 

PF04K61* 



+k2x{SC_target(B) ■ SC_ave(B)} xSPratio ...(4c) 



In the formulas, kl and k2 are predetermined coefficients. 
SC_target(R), SC_target(G) and SC_target(B) are skin color target values for 
5 each of the color components, and are values determined in advance. As 

described hereinafter, the values of the coefficients kl and k2 are, for example, 
set according to the image to be processed. The methods for setting the 
coefficients kl and k2 and the meaning of the color balance correction values 
AC(R), AC(G) and AC(B) are described hereinafter. 

10 FIG. 6 is a flow chart illustrating the processing procedure for color 

balance adjustment in the first embodiment. In Step SI the various color 
balance adjustment parameters described above, CB_ave, SPratio, SC_ave 
except the correction values AC(R), AC(G) and AC(B) are calculated. In Step S2, 
the color balance correction value calculation module 226 determines whether 

15 or not the average hue value for the high luminance pixels CB_ave(H) falls 

within a predetermined skin color range or skin color hue range. If the average 
hue value CB_ave(H) does not fall within the skin color range, the color 
balance correction value calculation module 226 performs computations in a 
first computation mode for calculating the color balance correction values 

20 using the skin color target values SC_target(R), SC_target(G) and 

SC_target(B) in Step S3. Specifically, the two coefficients kl and k2 in the 
formulas (4a) to (4c) above are both set to 1.0, for example. In this case, the 
formulas (4a) to (4c) above can be rewritten as the following formulas (5a) to 
(5c). 

25 AC(R)={CB_ave(Y) - CB_ave(R)} 

+{SC_target(R) - SC_ave(R)}xSPratio ...(5a) 
AC(G)={CB_ave(Y) - CB_ave(G)} 

+{SC_target(G) - SC_ave(G)}xSPratio ...(5b) 
AC(B)={CB_ave(Y) - CB_ave(B)} 
30 +{SC_target(B) - SC_ave(B)} xSPratio ...(5c) 



The first argument on the right-hand side of formulas (5a) to (5c) is 

- 10- 

PF04K61* 



the difference between the average luminance value of the high luminance 
pixels CB_ave(Y) and the average pixel value of the high luminance pixels for 
each of the color components CB_ave(R), CB_ave(G) and CB_ave(B). This has 
the effect of bringing the overall color of the image closer to achromatic when 
5 the color of the high luminance pixels is shifted from achromatic. The second 
argument on the right-hand side of formulas (5a) to (5c) multiplies the ratio of 
skin color pixels SPratio by the difference between the skin color target pixel 
values SC_target(R), SC_target(G) and SC_target(B) and the average pixel 
values of the skin color pixels SC_ave(R), SC_ave(G) and SC_ave(B). This 
10 second argument has the effect of bringing the average color of pixels having a 
color close to skin closer to the skin color target value when this is shifted from 
the skin color target value. Accordingly, if the overall image to be processed is 
subjected to color balance adjustment using the correction values AC(R), AC(G) 
and AC(B), which can be given by the formulas (5a) to (5c), it is possible to 
15 color balance the entire image to be processed while bringing the skin color 
pixels closer to the target skin color. 

Meanwhile, if it is determined in Step S2 that the average hue value 
CB_ave(H) falls within the skin color range, the color balance correction value 
calculation module 226 performs computations in a second computation mode 
20 for calculating the color balance correction values without using the skin color 
target values SC_target(R), SC_target(G) and SC_target(B) in Step S4. 
Specifically, in formulas (4a) to (4c) above, the first coefficient kl is set to a 
predetermined positive number other than zero, and the second coefficient k2 
is set to zero. For example, if the coefficient kl is set to 1.0 and the coefficient 
25 k2 is set to zero, the formulas (4a) to (4c) above can be rewritten as the 
following formulas (6a) to (6c). 

AC(R)={CB_ave(Y) - CB_ave(R)} ...(6a) 
AC(G)={CB,ave(Y) ■ CB_ave(G)} ...(6b) 
AC(B)={CB_ave(Y) ■ CB_ave(B)} ...(6c) 

30 

As described above, the right hand argument serves to bring the color 
of the overall image closer to achromatic when the color of the high luminance 
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pixels is shifted from achromatic, allowing for suitable color balancing of the 
overall image. Note that the reason for which the coefficient k2 in the second 
argument on the right-hand side of the formulas (4a) to (4c) is set to zero if it is 
determined that the average hue value of the high luminance pixels 
5 CB_ave(H) falls within the skin color range, is to prevent the correction value 
from becoming excessively large as a result of the second argument on the 
right-hand side. 

However, the color balance correction values AC(R), AC(G) and AC(B) 
may be calculated according to formulas (4a) to (4c) or formulas (5a) to (5c), 

10 whether or not the average hue value of the high luminance pixels CB_ave(H) 
falls within the skin color range. In this case, the coefficient kl in the first 
argument on the right hand side may be set to zero. 

When the color balance correction values AC(R), AC(G) and AC(B) 
have been calculated, a color balance adjustment execution module 228 

15 executes the color balance adjustment in Step S5 of FIG. 6, so as to generate 
the corrected image data Dcr (FIG. 3). FIG. 7 is a graph showing the color 
balance adjustments performed using the correction values AC(R), AC(G) and 
AC(B). The color balance adjustment is a process whereby gradation correction 
or tone curve correction is performed for each of the RGB color components in 

20 the data for the image to be processed Dorg. The dotted line indicates the 
original characteristics. The correction values AC(R), AC(G) and AC(B) 
calculated by the procedure described above, correspond to the amounts by 
which the output value is shifted from the original characteristics at a 
reference input value Dref. The average luminance value of the high 

25 luminance pixels CB_ave(Y) described above can, for example, be used as the 
reference input value Dref, or a predetermined constant (for example, 192) can 
be used. In the example in FIG. 7, the tone curves for each of the color 
components are determined by specifying the correction values AC(R), AC(G) 
and AC(B) for each one of the RGB color components with respect to the 

30 reference input value Dref. However, other setting methods may be used. 

As described above, in the first embodiment, color balance correction 
values for the image to be processed are determined using the skin color target 
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values, making it possible to mitigate the phenomenon wherein pixels having a 
desirable skin color are unduly changed by the color balance adjustment. 

B. Second Embodiment 

5 FIG. 8 is a flow chart illustrating the processing procedure for color 

balance adjustment in the second embodiment. The only difference with 
respect to the processing procedure in the first embodiment shown in FIG. 6 is 
that Step S2 is replaced by two steps Sll and S12, with the exception of which 
the constitution and operation are the same as those of the first embodiment. 

10 In Step Sll the color balance correction value calculation module 226 

obtains the photographic scene type from the image data file MF (FIG. 2). In 
Step S12, a determination is made as to whether the photographic scene type 
represents " portrait," and if this does represent " portrait," the color balance 
correction values are calculated using the skin color target values in Step S3. 

15 Meanwhile, if the photographic scene type is not " portrait," the color balance 
correction values are calculated without using the skin color target values in 
Step S4. 

The photographic scene type is set by the user when taking a 
photograph with the digital camera 100. Because skin color is emphasized in 

20 the photographic scene type of" portrait," it is preferable that the color balance 
adjustment values be found using the skin color target values in Step S3. Note 
that whether or not the photographic scene is of a portrait may be determined 
from information other than the photographic scene type. That is to say, 
whether or not the photographic scene is of" portrait " may be determined by 

25 any information indicating the type of photographic scene in the image (this is 
referred to as "photographic scene type information"). 

C. Variants 

Note that the present invention is in no way limited to the examples 
30 and embodiments described above and can be embodied in various modes 
without departing from the scope or gist of the invention, wherein the 
following modifications are, for example, possible. 
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CI, Variant l: 

In place of the various formulas described above, various methods 
may be employed to calculate color balance correction values based on the ratio 
of pixels of a specific color and the average pixel value of the pixels of the 
5 specific color. For example, the second argument on the right-hand side of 

formulas (4a) to (4c) was a method wherein the skin color pixel ratio SPratio is 
multiplied by the difference between the target pixel values for skin color 
SC_target(R), SC_target(G) and SC_target(B) and the average pixel values for 
skin color pixels SC_ave(R), SC_ave(G) and SC_ave(B), but in place of this, 

10 methods can be used that include the squares or square roots of these values. 
However, it is preferable that the color balance correction value have a 
component that increases with increases in the difference between the target 
pixel values for skin color SC_target(R), SC_target(G) and SC_target(B) and 
the average pixel values for skin color pixels SC_ave(R), SC_ave(G) and 

15 SC_ave(B), and that increases with increases in the skin color pixel ratio 

SPratio, as in the second argument on the right-hand side of formulas (4a) to 
(4c). 

Note that, in each of the embodiments described above, a ratio of 
pixels of a specific color is employed when finding the color balance correction 

20 values, but the color balance correction values may be found without 

employing a ratio of pixels of a specific color. For example, the color balance 
correction values may be found based on the average pixel values of pixels of a 
specific color. Furthermore, the color balance correction values may be found 
based on the average pixel values of pixels of a specific color and the average 

25 pixel values of high luminance pixels. 
C2. Variant 2: 

In each of the embodiments described above, the only image quaUty 
adjustment processing performed is color balance adjustment, but various 
other t5T)es of image quality adjustment processing may be performed in 
30 addition to this. For example, memory color processing, sharpness 

enhancement processing and the like may be performed. Note that the memory 
color processing is processing wherein, only for pixels having a color close to a 
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memory color (specific chromatic color), the color of these pixels is brought 
closer to its target color. 

INDUSTRIAL APPLICABILITY 
5 The present invention is applicable to computers, cameras, image 

display devices and the like that perform image color balance adjustment. 
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